File systems store files and store information about files. The information stored in files may be referred to as data. The information about files may be referred to as metadata. The collection of files and metadata may be referred to collectively as a file system. When the data in a file changes, the metadata about that file in the file system may also be changed. For example, if the contents of a file are changed, the metadata may be updated to reflect the time at which the change was made and by whom the change was made. The file system metadata may be stored in a set of data structures including inodes, btrees, directory structures, and others.
Over time, a file system may develop inconsistencies or other errors. Therefore, tools may be provided for checking the consistency or state of a file system. For example, the system utility fsck (“file system check”) can be used to check the consistency of a file system. Conventional file system checkers may make two passes through the file system. For example, a first pass may be performed in a read-only mode to discover problems in the file system. Once the problems are identified, a second pass may be performed in a read-write mode to attempt to actually fix the problems discovered during the read-only pass. Conventionally, fixing the problems may include writing new blocks to the file system, deleting blocks from the file system, editing existing blocks or data structures, or other actions.
Conventionally, the two pass approach made sense because it was a good idea to identify problems in the file system and to identify what could be done about those problems before deciding to remedy or ignore the errors. Unfortunately, making two complete passes through the file system may have consumed an unacceptable amount of time. The amount of time may have been unacceptable because the file system may have been made unavailable to users while the two passes were performed. Additionally, the act of making a first pass through the file system in read-only mode may have produced artifacts or false errors when reporting problems. By way of illustration, during the first pass through the file system, a conventional file system checker may identify three problems. The three problems may be recorded and presented to the second pass for repair. However, during the second pass, fixing the first problem may also fix the third problem. Therefore, the third problem may become an artifact that the second pass attempts to fix even though the problem no longer exists after the first problem is fixed.
The two pass approach may produce artifacts or other errors due to the sequence with which discovered problems may be fixed. Conventionally, malformed modes may be fixed first, then btrees associated with the malformed modes may be fixed, and then directory information associated with the btrees may be repaired. In the read-only pass, the errors may be discovered but not repaired, because the first pass is read-only and making repairs may require writing to the file system. Then, during the second pass, errors that were detected during the first pass and scheduled for fixing during the second pass may no longer exist when the second pass gets to the scheduled fix. Therefore, later btree and directory checks might report false problems.
Even worse, the act of making a scheduled fix of an error detected during the first pass may produce a new error that is not scheduled for repair. Thus, the second pass may complete all the fixes identified in the first pass but may not fix a new error or inconsistency produced by fixing a problem identified in the first pass. Making a change to a file may require the file system to perform several operations. Ideally, the operations would either be “all done” or “none done.” Undesirable conditions may arise if a series of operations are only partially done. Thus, a file system may choose to treat a series of operations as a transaction. Example transactions may include allocating space for a file, creating a file, updating a file, deleting a file, or other operations. While the file system may choose to treat operations as a transaction, an underlying operating system or other actor (e.g., storage system) may only be able to guarantee that individual members of the series of operations are performed as atomic operations. When a transaction does not complete, inconsistencies may be produced. A transaction may not complete as a result of different error conditions.
Therefore, file systems may use a journal to help support correctly performing a series of operations as a single file system transaction. The journal may be, for example, a disk-based structure that can store information about operations to be performed to transition a file system from a first state to a second state. The journal may be used to store a complete representation of the set of operations that are to be completed for the file system transaction. For example, the journal may store a linear sequence of underlying operations that are to be performed as part of the file system transaction. Conventionally it was thought that once the set of operations to be performed were written in the journal, the set of operations could be started safely in the knowledge that if something went wrong, it may have been possible to back out of the set of operations using the information stored in the journal.
One issue with file systems arises due to the difference in latency between memory and non-memory (e.g., disk, tape) storage. This latency can produce conditions where changes made in one area (e.g., memory) are out of sync with changes made in another area (e.g., disk). Additionally, this latency motivates a file system to store in memory changes that are to be made to data on disk and then to make the actual changes on disk at a later time. For example, a series of reads and writes to a file may be made virtually in memory and then only made physically on disk at a later time. While this delayed update approach may solve one problem associated with excessive random input/output (i/o), it may produce another problem associated with memory and disk being out of synchronization. When the memory and disk get out of synchronization, then the file system may become inconsistent. The file system metadata may indicate that a change has been made, and that change may have been performed in memory, but the actual underlying data on disk may not have been changed.
Running a conventional file system checker on a file system that is mounted for read/write operations may produce data corruption or loss. Therefore, a file system is typically checked while the file system is un-mounted, while the file system is mounted read-only, or while the file system is in a special maintenance mode that limits the risk of damage. Thus, running a conventional file system checker may have required the file system to be taken out of operation. When 24/7/365 operations are expected or required, it may be difficult, if even possible at all, to run a conventional file system checker, which may lead to small inconsistencies or errors in the file system developing into potentially catastrophic errors.